Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proof Validation] Move proof validation to session settlement #703

Merged
merged 38 commits into from
Jul 25, 2024

Conversation

Olshansk
Copy link
Member

@Olshansk Olshansk commented Jul 23, 2024

Summary

⚠️ WARNING: THIS IS NOT AN EASY PR TO REVIEW ⚠️

  • A lot of components are moving aroud
  • This is sensitive since it's related to proof validation
  • Best effort is all that's expected

What was changed?

  • Move proof validation from SubmitProof in the msgServer handler to SessionSettlement during the EndBlocker.

Why was it changed?

  1. RPC requests should be quick, lightweight and only do basic validation
  2. Validators are the ones responsible for the heavy processing & validation during state transitions
  3. This creates an opportunity to slash suppliers who submit false proofs, whereas
    they can keep retrying if it takes place in the SubmitProof handler.

Other:

  • A few other changes related to test & test helper improvement were made
  • Some improvements to the claim & proof creation logic were made

Issue

Type of change

Select one or more:

  • New feature, functionality or library
  • Bug fix
  • Code health or cleanup
  • Documentation
  • Other (specify)

Testing

  • Unit Tests: make go_develop_and_test
  • LocalNet E2E Tests: make test_e2e
  • DevNet E2E Tests: Add the devnet-test-e2e label to the PR.

Sanity Checklist

  • I have tested my changes using the available tooling
  • I have commented my code
  • I have performed a self-review of my own code; both comments & source code
  • I create and reference any new tickets, if applicable
  • I have left TODOs throughout the codebase, if applicable

Summary by CodeRabbit

  • New Features

    • Introduced an expiration reason for claims to enhance clarity on claim status.
    • Added new error handling for non-existent accounts in proof submissions.
    • Implemented comprehensive proof validation and related tests to improve reliability.
    • Expanded interfaces for account and proof management capabilities.
  • Bug Fixes

    • Enhanced error handling for account retrieval to prevent null-related issues.
  • Tests

    • Improved test structure for claim settlement and proof validation.
  • Documentation

    • Updated comments and structured the code for better readability and maintainability.

@Olshansk Olshansk added protocol General core protocol related changes on-chain On-chain business logic labels Jul 23, 2024
Olshansk and others added 6 commits July 23, 2024 14:38
- This is a review of #690
- Creating a custom PR to make it easier to track the changes for future reference
---

Co-authored-by: Bryan White <[email protected]>
@Olshansk Olshansk changed the base branch from refactor/difficulty/target-hash-review to refactor/difficulty/target-hash July 24, 2024 01:29
@Olshansk Olshansk marked this pull request as ready for review July 24, 2024 01:42
Copy link
Contributor

@red-0ne red-0ne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tough one, reviewed twice, but not that many change requests. Great job!

x/tokenomics/keeper/settle_pending_claims.go Outdated Show resolved Hide resolved
x/tokenomics/keeper/settle_pending_claims.go Outdated Show resolved Hide resolved
Comment on lines 89 to 90
isProofValid, err = k.proofKeeper.IsProofValid(ctx, &proof)
if !isProofValid || err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that IsProofValid always returns false when there's an error. What about renaming this to EnsureValidProof and only return err?

Suggested change
isProofValid, err = k.proofKeeper.IsProofValid(ctx, &proof)
if !isProofValid || err != nil {
if err := k.proofKeeper.EnsureValidProof(ctx, &proof); err != nil {

Copy link
Member Author

@Olshansk Olshansk Jul 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great suggestion! Done!!

x/tokenomics/keeper/settle_pending_claims.go Show resolved Hide resolved
x/tokenomics/keeper/keeper_settle_pending_claims_test.go Outdated Show resolved Hide resolved
x/proof/keeper/proof_validation.go Show resolved Hide resolved
x/proof/keeper/proof_validation.go Show resolved Hide resolved
x/proof/keeper/proof_validation.go Show resolved Hide resolved
x/proof/keeper/proof_validation_test.go Outdated Show resolved Hide resolved
x/proof/keeper/proof_validation_test.go Outdated Show resolved Hide resolved
Base automatically changed from refactor/difficulty/target-hash to main July 24, 2024 20:26
Copy link

coderabbitai bot commented Jul 24, 2024

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

Commits

Files that changed from the base of the PR and between 54b0261 and 9e8e8e8.

Files selected for processing (10)
  • proto/poktroll/shared/service.proto (1 hunks)
  • testutil/keeper/tokenomics.go (2 hunks)
  • x/proof/keeper/msg_server_create_claim.go (5 hunks)
  • x/proof/keeper/msg_server_submit_proof.go (5 hunks)
  • x/proof/keeper/proof_validation.go (1 hunks)
  • x/proof/keeper/proof_validation_test.go (1 hunks)
  • x/tokenomics/keeper/keeper_settle_pending_claims_test.go (7 hunks)
  • x/tokenomics/keeper/settle_pending_claims.go (3 hunks)
  • x/tokenomics/keeper/settle_session_accounting_test.go (1 hunks)
  • x/tokenomics/types/expected_keepers.go (4 hunks)
____________________________________________________________________________________________________________________________
< Developing tolerance for imperfection is the key factor in turning chronic starters into consistent finishers. - Jon Acuff >
----------------------------------------------------------------------------------------------------------------------------
 \
  \   (\__/)
      (•ㅅ•)
      /   づ

Walkthrough

The recent changes enhance several components within the codebase, primarily focusing on improving error handling and validation processes. Key modifications include the introduction of new fields and methods for managing claim expiration reasons, refining account management, and restructuring session handling logic. Additionally, test suites have been updated to improve reliability and clarity, ensuring that the overall functionality of claim settlements, proof validations, and account operations remain robust and effective.

Changes

Files Change Summary
api/poktroll/tokenomics/event.pulsar.go, proto/poktroll/tokenomics/event.proto Added ExpirationReason field and ClaimExpirationReason enum to enhance claim expiration handling.
pkg/client/query/accquerier.go, x/proof/types/account_query_client.go Improved error handling in account retrieval methods to prevent null dereference errors.
x/proof/keeper/msg_server_create_claim.go, x/proof/keeper/msg_server_submit_proof.go Streamlined claim creation and proof submission logic with updated validation and error handling mechanisms.
x/proof/keeper/proof_validation.go, x/proof/keeper/proof_validation_test.go Implemented comprehensive proof validation and corresponding tests to ensure integrity and reliability.
x/tokenomics/keeper/settle_pending_claims.go, x/tokenomics/keeper/settle_session_accounting_test.go Enhanced claim settlement logic with detailed proof validation and improved test structure.
x/tokenomics/types/expected_keepers.go Expanded interfaces for AccountKeeper and ProofKeeper with new methods for account and proof management.
testutil/keeper/tokenomics.go, testutil/testtree/tree.go Introduced session management functionality and utility functions for managing session trees in tests.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Outside diff range, codebase verification and nitpick comments (11)
x/tokenomics/keeper/settle_pending_claims.go (3)

83-83: Clarify the purpose of proofIsRequired.

The new variable proofIsRequired encapsulates the condition of whether a proof is necessary. This improves code readability.


94-95: Clarify the expiration reason for missing proofs.

The comment clarifies that the claim should expire if a proof is required but unavailable.


128-131: Clarify the conditions for settling claims.

The comments clarify the conditions under which claims are settled, improving code readability.

api/poktroll/tokenomics/event.pulsar.go (1)

23-23: Add a comment to describe the new field.

The expiration_reason field is newly added and should be documented for clarity.

fd_EventClaimExpired_expiration_reason protoreflect.FieldDescriptor // Reason for the claim expiration
x/proof/keeper/proof_validation.go (7)

57-57: Consider using a more descriptive logger message.

The logger message "ValidateProof" could be more descriptive to reflect the function name IsProofValid.

- logger := k.Logger().With("method", "ValidateProof")
+ logger := k.Logger().With("method", "IsProofValid")

211-212: Typographical error in the comment.

There is a typographical error in the comment: "to to" should be "to".

- // The RelayMiner has to wait until the submit claim and proof windows is are open
- // in order to to create the claim and submit claims and proofs, respectively.
+ // The RelayMiner has to wait until the submit claim and proof windows are open
+ // in order to create the claim and submit claims and proofs, respectively.

238-239: Clarify TODO comment.

The TODO comment should be more specific about what needs to be investigated.

- // TODO_BETA: Investigate "proof for the path provided does not match one expected by the on-chain protocol"
- // error that may occur due to block height differing from the off-chain part.
+ // TODO_BETA: Investigate the error "proof for the path provided does not match one expected by the on-chain protocol"
+ // which may occur due to block height differing from the off-chain part.

263-263: Clarify NB comment.

The NB comment should be more specific about what is being asserted.

- // NB: no need to assert the testSessionId or supplier address as it is retrieved
- // by respective values of the given proof. I.e., if the claim exists, then these
- // values are guaranteed to match.
+ // NB: No need to assert the session ID or supplier address as they are retrieved
+ // by the respective values of the given proof. If the claim exists, then these
+ // values are guaranteed to match.

279-279: Improve error message consistency.

The error message for ErrProofInvalidSessionStartHeight should be consistent with other error messages.

- return nil, types.ErrProofInvalidSessionStartHeight.Wrapf(
- "claim session start height %d does not match proof session start height %d",
- claimSessionHeader.GetSessionStartBlockHeight(),
- proofSessionHeader.GetSessionStartBlockHeight(),
- )
+ return nil, types.ErrProofInvalidSessionStartHeight.Wrapf(
+ "claim session start height (%d) does not match proof session start height (%d)",
+ claimSessionHeader.GetSessionStartBlockHeight(),
+ proofSessionHeader.GetSessionStartBlockHeight(),
+ )

342-342: Typographical error in the error message.

There is a typographical error in the error message: "sessionHeaders" should be "session headers".

- "sessionHeaders service names mismatch expect: %q, got: %q",
+ "session headers service names mismatch; expected: %q, got: %q",

378-378: Typographical error in the comment.

There is a typographical error in the comment: "the the" should be "the".

- // verifyClosestProof verifies the the correctness of the ClosestMerkleProof
- // against the root hash committed to when creating the claim.
+ // verifyClosestProof verifies the correctness of the ClosestMerkleProof
+ // against the root hash committed to when creating the claim.

Comment on lines +102 to +103
// TODO_TECHDEBT: add a test case such that we can distinguish between early
// & late session end block heights.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder: Add test cases for early & late session end block heights.

The TODO comment indicates missing test cases for distinguishing between early and late session end block heights.

Do you want me to generate the test cases or open a GitHub issue to track this task?

Comment on lines +333 to +334
// TODO_TEST(community): expand: test case to cover each session header field.
desc: "relay request session header must match proof session header",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder: Expand test case to cover each session header field.

The TODO comment indicates missing test cases for each session header field.

Do you want me to generate the test cases or open a GitHub issue to track this task?

Comment on lines +382 to +383
// TODO_TEST: expand: test case to cover each session header field.
desc: "relay response session header must match proof session header",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder: Expand test case to cover each session header field.

The TODO comment indicates missing test cases for each session header field.

Do you want me to generate the test cases or open a GitHub issue to track this task?

Comment on lines +485 to +487
desc: "relay request signature is valid but signed by an incorrect application",
newProof: func(t *testing.T) *prooftypes.Proof {
t.Skip("TODO_TECHDEBT(@bryanchriswhite): Implement this")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implement the test case for invalid relay request signature signed by an incorrect application.

The test case is marked as TODO and needs to be implemented.

Do you want me to generate the test case or open a GitHub issue to track this task?

Comment on lines +542 to +544
desc: "relay response signature is valid but signed by an incorrect supplier",
newProof: func(t *testing.T) *prooftypes.Proof {
t.Skip("TODO_TECHDEBT(@bryanchriswhite): Implement this")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implement the test case for valid relay response signature signed by an incorrect supplier.

The test case is marked as TODO and needs to be implemented.

Do you want me to generate the test case or open a GitHub issue to track this task?

// TODO_FOLLOWUP(@olshansk, #690): Get the difficulty associated with the service
params := k.GetParams(ctx)
relayDifficultyTargetHash := params.RelayDifficultyTargetHash
if relayDifficultyTargetHash == nil || len(relayDifficultyTargetHash) == 0 {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Omit nil check; len() for []byte is defined as zero.

The nil check for relayDifficultyTargetHash is redundant as len() for []byte is defined as zero.

- if relayDifficultyTargetHash == nil || len(relayDifficultyTargetHash) == 0 {
+ if len(relayDifficultyTargetHash) == 0 {
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if relayDifficultyTargetHash == nil || len(relayDifficultyTargetHash) == 0 {
if len(relayDifficultyTargetHash) == 0 {
Tools
GitHub Check: go-test

[failure] 157-157:
S1009: should omit nil check; len() for []byte is defined as zero (gosimple)

@Olshansk Olshansk requested a review from red-0ne July 24, 2024 21:05
Copy link
Contributor

@red-0ne red-0ne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a linter error that is making CI to fail
image

Otherwise LGTM!

@Olshansk Olshansk merged commit 0049068 into main Jul 25, 2024
10 checks passed
@Olshansk Olshansk deleted the pr_690_followups branch July 25, 2024 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
on-chain On-chain business logic protocol General core protocol related changes
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

3 participants